Papers with neural models of language
Correlating Neural and Symbolic Representations of Language (P19-1)
Copied to clipboard
| Challenge: | a popular technique for analyzing neural representations involves predicting information of interest from the activation patterns. |
| Approach: | They propose to use Representational Similarity Analysis and Tree Kernels to quantify how strongly activation patterns correspond to symbolic representations. |
| Outcome: | The proposed methods show that they exhibit the expected pattern of results on a synthetic language. |
Unsupervised Learning of Hierarchical Conversation Structure (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Goal-oriented conversations often have sub-dialogue structure, but it can be domain-dependent . Increasingly, language understanding applications involve conversational speech and text . |
| Approach: | They propose an unsupervised approach to learning hierarchical conversation structure . they use turn and sub-dialogue segment labels to decode the structure based on dialogue acts and subtasks . |
| Outcome: | The proposed approach improves neural models for three conversation-level understanding tasks. |
A Tale of a Probe and a Parser (2020.acl-main)
Copied to clipboard
| Challenge: | researchers train supervised models to extract linguistic structure from output of another model . supervised model can be used to perform tasks such as part-of-speech tags or dependency trees . |
| Approach: | They compare a structural probe to a more traditional parser with a lightweight parameterisation. |
| Outcome: | The structural probe outperforms a traditional parser on seven of nine languages . the researchers found that the model outperformed the parsers by 11.1 points . |